“You two, join Green”: A Human Multi-Robot Interface for Creating and Commanding Teams of Robots

We present a novel multimodal system for creating and commanding groups of robots from a population. Extending our previous work on dynamically creating groups of robots using face engagement and voice commands, we show that we can identify an individual or a group of robots using haptic stimuli, and name them using a voice command (e.g. “You two, join Green”). Subsequent commands can be addressed to the same robot(s) by name. 1. MOTIVATION AND BACKGROUND In multi-robot systems performing complex missions, it is expected to be beneficial to integrate human cognitive capabilities for high level supervisory control. A key enabler to allowing a human operator to control a large population of robots is the ability to select and/or command multiple robots in parallel. Single or multiple robots can be selected and identified as a group, and the whole group is commanded with a single interaction. This is a challenging problem since each robot must decide if the user is paying attention to it or its peers. We have been working on methods for a single human operator to dynamically select and command individual or groups of robots from a population. The motivation for these interface designs is to increase the span of control (one operator to multiple robots) which can broadly benefit in several applications (e.g. coordinated exploration). Olsen and Wood [2] introduced the concept of Fan-out which posits a model-based upper-bound on the number of independent homogeneous unmanned vehicles (UVs) that a single human can interact with. Fan-out is defined as the ratio of activity time (the time a robot operates autonomously), to interaction time (the expected amount of time that a human must interact with one or a group of robots). We have previously shown that interaction time can be reduced by creating groups of robots and doing concurrent interactions that exploit locality [4]. This can be led to an increase in the robot/human Fan-out ratio. We focus on designing spatially embedded interfaces for human multi robot systems (HMRS) that provide an efficient and natural way to form and interact with groups of robots. By spatially embedded interfaces, we mean those in which the interaction occurs directly between human and robots in a shared physical workspace, mediated at least partly by the robot’s sensors. This is an important class of interfaces, since it allows for what can be called “natural” means of communication such as speech, gestures, haptic and face engagement. We have developed interfaces using these modalities, for systems that require dynamic task allocation, team composition and team re-composition. We have previously shown that the user can dynamically create groups of desired number of robots, by verbally announcing the desired number of robots and simply looking directly at them [3]. The robots wirelessly exchange their independent observations of the user’s face to determine which robots are selected. The team can then be commanded as a This paper was nominated in the ”New Technology Foundation Award for Entertainment Robots and Systems” at IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’13). unit with e.g. “Take off!”. The video demonstration of the system is available here: http://www.youtube.com/watch?